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The MAILING DATE of this communication appears on the cover sheet with the correspondence address » 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- tf the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )I3 Responsive to cx)mmunication(s) filed on 10 October 2000 . 
2a)D This action is FINAL. 2b)K This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) (3 Claim(s) 7-30 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) E3 Claim(s) 1-30 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10) D The drawing(s) filed on is/are: a)D accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

1 1) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 
a)D All b)D Some * c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

2.D Certified copies of the priority documents have been received in Application No. . 



3.Q Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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DETAILED ACTION 



This action is responsive to communications. Application filed 10/10/2000. 



Claims 1-30 are pending in the case. Claims 1,13, and 22, are independent claims. 



Claim Rejections - 35 (JSC §103 



3. The following is a quotation of 35 U.S.C 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

4. Claims 1-30 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Murakami et al., USPN 6,108,674 filed (1/16/1997) in view of Lieberman, 'Autonomous 
Interface Agents', March 1997, in further view of Katz et aL, USPN 5,404,295 filed 
(1/4/1994). 

In reference to independent claim 1, Murakami teaches: 

A central processing unit is provided for retrieving, from the image classification data 
memory, image identification data indicative of all the respective image classification data input 
by the keyboard (compare to "continuously monitoring text typed by the user into the 
application to isolate the context expressed by the text" and "running a user application in 
which text is entered by a user". See column 3, lines 50-67 and column 4, lines 1-67. 

The Murakami does not explicitly disclose continuously monitoring text typed by the 
user into the application to isolate the context expressed by the text; however, Lieberman 
teaches, An agent may observe many user inputs, over a long period of time, before deciding to 
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take a single action, or a single user input may launch a series of actions on the part of the agent. 
See column 5, lines 43-67. It would have been obvious to one of ordinary skill in the art, having 
the teachings of Lieberman and Murakami before him at the time the invention was made, to 
modify the keyword image matching techniques of taught by Murakami to include the direct- 
manipulation actions on objects as disclosed in Lieberman, because it would supply the 
autonomous agent a continual supply of desired output image data based on selected keywords of 
textual files. 

Both Murakami and Lieberman do not explicitly disclose utilizing the context to provide 
suggested annotations to the user for the matched images; however Katz teaches annotations, 
which could be in the form of questions, assertions or noun phrases or some combination or 
collection thereof See column 5, lines 50-67 and column 6, lines 1-54. The methods suggest the 
annotation of text within databases from which material may be retrieved utilizing the teachings 
of the invention include ones containing graphics, images, etc. It would have been obvious to one 
of ordinary skill in the art, having the teachings of Lieberman, Murakami, and Katz before him at 
the time the invention was made, to modify the continuous monitoring methods of Lieberman 
and the image matching/retrieval methods of Murakami, to include the image annotation 
methods of Katz, because it would have provided the author the added benefit of having a more 
useful for pattern matching purposes within an image retrieval system. 
In reference to dependent claim 2, Murakami teaches: 

The keyword image matching techniques as taught by Murakami allow a user to inset 
selected ones of the matched images into the text of the application automatically. See column 8, 
lines 32-57. 
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In reference to dependent claim 3-5, Murakami teaches: 

Since this character string is also contained in the image classification data of the 
appropriate file name, the CPU stores the appropriate file name in the retrieval result work 
memory. See column 8, lines 30-45. 
In reference to dependent claim 6, Murakami teaches: 

A time register records the current time (year, month, date, hour, minute, second). The 
CPU accesses an illustration memory on the bases of the recorded month data. See column 10, 
lines 25-35. 

In reference to dependent claim 7, Murakami teaches: 

The position where the illustration is inserted is the current position of the cursor. See 
column 5, lines 13-17. 

In reference to dependent claim 8, Murakami teaches: 

Murakami discloses character strings being received from an input device; however, the 
reference does not explicitly disclose adding other annotations to the matched images. Katz 
teaches methods, which suggest the annotation of text within databases from which material may 
be retrieved utilizing the teachings of the invention include ones containing graphics, images, 
etc. It would have been obvious to one of ordinary skill in the art, having the teachings of 
Lieberman, Murakami, and Katz before him at the time the invention was made, to modify the 
continuous monitoring methods of Lieberman and the image matching/retrieval methods of 
Murakami, to include the image annotation methods of Katz, because it would have provided the 
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author the added benefit of having a more useful for pattern matching purposes within an image 
retrieval system. 



In reference to dependent claim 9-12, Murakami teaches: 

The retrieval of images based on 3 different keywords. The utilization of the three 
different keywords provides the suggestion of image retrieval ranking. See column 19-31. 
In reference to claims 13-21, the claims recite similar limitations to those of claims 1-13, and in 
further view of the following, are rejected along the same rationale. 

In reference to claims 22-30, the claims reflect the system used for performing the methods as 
claimed in claims 1-13, and in further view of the following, is rejected along the same rationale. 

Conclusion 

5. The prior art made of record and not relied upon is considered pertinent to applicants 
disclosure. 

Windle USPN 6,686,970 filed 1 0/5/98 

Any inquiry concerning this communication or earlier communications from the 

examiner should be directed to Matthew J. Ludwig whose telephone number is 703-305-8043. 

The examiner can normally be reached on 8:00am-5:00pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 

supervisor, Heather Herndon can be reached on 703-308-5186. The fax phone number for the 

organization where this application or proceeding is assigned is 703-872-9306. 
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Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 
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